similarity challenge
Producing augmentation-invariant embeddings from real-life imagery
Papadakis, Sergio Manuel, Addicam, Sanjay
This article presents an efficient way to produce feature-rich, high-dimensionality embedding spaces from real-life images. The features produced are designed to be independent from augmentations used in real-life cases which appear on social media. Our approach uses convolutional neural networks (CNN) to produce an embedding space. An ArcFace head was used to train the model by employing automatically produced augmentations. Additionally, we present a way to make an ensemble out of different embeddings containing the same semantic information, a way to normalize the resulting embedding using an external dataset, and a novel way to perform quick training of these models with a high number of classes in the ArcFace head. Using this approach we achieved the 2nd place in the 2021 Facebook AI Image Similarity Challenge: Descriptor Track.
The Image Similarity Challenge and data set for detecting image manipulation
We also worked with trained third-party annotators to manually transform a smaller subset of the images to ensure we have even more selections representative of the way a human user would transform images. The annotators used image manipulation software GIMP to manually alter images in diverse ways that we cannot easily automate, for example handwriting or drawing on the images or cropping to leave only the part of the image most salient to the human eye. The Image Similarity Challenge invites participants to test their image matching techniques on the Image Similarity data set. More information for researchers is available here, and the accompanying paper is available here. For researchers considering attending NeurIPS 2021 in December, we're also pleased to announce that the Image Similarity Challenge has been accepted for the NeurIPS 2021 competition track, where we will be announcing the winners of this challenge (The competition is subject to official rules.